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Q @ Modelling Dynamic e-Business Applications Using Historical Performance Data 

D.Bacigalupo, J.D.Tumer, S.AJarvis, G.R.Nudd 

19th Annual UK Performance Engineering Workshop (UKPEW03) 

University of Warwick UK 9- 10 July 2003 

Distributed e-business application platforms such as the web services framework will require 
sophisticated workload management infrastructures for the matching of client requests to the 
most appropriate resources. This paper introduces a novel dynamic predictive framework that 
uses both historical data and analj^ical performance modelling to extrapolate a request's 
predicted performance imder varying workloads and conditions on different systems. Such a 
framework can facilitate a workload manager in maximising the extent to which service level 
agreements can be met. Initial results are presented analysing how the response times of an e- 
business benchmark can be predicted at different system loads and on different application 
servers. 

^ ^ A Dynamic Predictive Framework for e-business Workload Management 

D.Bacigalupo, J.D.Tumer, S.A.Jarvis, G.R.Nudd 

7th World Multiconference on Systemics, Cybernetics and Informatics (SCI2003) Performance 
of Web Services Invited Session, 
Orlando, Florida USA, 27-30 July 2003 

Distributed e-business application platforms such as the web services framework will require 
sophisticated workload management infrastructures for the routing of cUent requests to the most 
appropriate services. This paper introduces a novel and dynamic predictive framework that uses 
both analytical performance modeling and historical data to extrapolate a request's predicted 
performance under varying workloads and conditions. Such a framework can facilitate a routing 
algorithm in meeting quality of service-based metric requirements. An e-business benchmark is 
used as a demonstrator to provide an insight into the techniques used in predicting the response 
time of requests under varied workload conditions, 

^ ^ Grid Resource Management Information Services for Scientific Computing 

H. N, Lim Choi Keung, L. Wang, D. P. Spooner, S. A. Jarvis, W. Jie and G. R. Nudd 
International Conference on Scientific and Engineering Computation 2002 (IC-SEC 2002) 
Raffles City Convention Centre, Singapore, December 3-5 2002. 

Scientific applications typically have considerable memory and processor requirements. 
Nevertheless, even with today's fastest supercomputers, the power of available resources falls 
short of the demands of large, complex simulations and analysis. The solution fortunately lies in 
the emergence of irmovative resource environments such as Grid computing. This paper 
addresses issues concerning the discovery and monitoring of resources across multiple 
administrative domains, which can be harnessed by scientific applications. Grid Information 
Services are an important infi:astructure in computational Grid environments as they provide 
important information about the state and availability of resources. We will present how the 
Globus Monitoring and Discovery Service (MDS) architecture can be extended to offer a 
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transparent, unified view of Grid resources. Local resource managers ultimately possess the 
potential to influence where scientific jobs are processed, depending on the availability and load 
of their scheduler. Consequently, information about the structure and the state of schedulers 
should be provided to Grid brokers, which will then decide where to schedule the applications. 
This paper will present the way in which low-level scheduling information is collated from 
multiple sources and is incorporated into the unified Grid information view. The existence and 
availability of Grid resource information also allows reliable application performance prediction 
to be carried out using the Warwick PACE (Performance Analysis and Characterisation 
Environment) system. Thus, Grid Information Services are a key component in the efficient 
execution of scientific applications on Grid computing architectures. 

"3 @ Localised Workload Management Using Performance Prediction and QoS Contracts. 

D. P. Spooner, J. Cao, J. D. Turner, H. N. Lim Choi Keung, S. A. Jarvis, and G. R. Nudd 
Eighteenth Annual UK Performance Engineering Workshop (UKPEW 2002) 
University of Glasgow, UK, July 10-11 2002. 

In this research, we approach the problem of grid workload management with the development 
of a multi-tiered architecture tiiat employs performance prediction and a hierarchy of 
homogeneous brokers to meet quality-of-service contracts and improve resource efficiency. By 
coupling application performance data with iterative hexuistic algorithms, the framework is able 
to balance the processes of minimising run-to-completion time (makespan) and idle time, whilst 
adhering to service deadlines on a per-task basis. This paper provides a brief introduction to the 
architecture, and then focuses on the lowest tier which is responsible for scheduling tasks on 
grid-enabled resources. Experimental results demonstrate that these components are able to 
adjust to the conflicting requirements of system metrics and QoS contracts in real-time, using 
the just-in-time capabilities of the PACE prediction environment and the iterative workload 
management algorithms. 

^ ^ Grid Information Services using Software Agents 

H. N. Lim Choi Keung, J. Cao, D. P. Spooner, S. A. Jarvis, and G. R. Nudd 
Eighteenth Annual UK Performance Engineering Workshop (UKPEW* 2002) 
University of Glasgow, UK, July 10-11 2002, pp. 187-198. 

Computational grids allow large-scale, pervasive and consistent sharing of geographically 
dispersed resources. Their inherent nature incorporates issues including the discovery of 
resources located in different administrative domains, predicting the performance of those 
resources and monitoring their behaviour. The Monitoring and Discovery Service (MDS), one 
of the pillars provided by the Globus toolkit, can be used to offer Grid information services to 
an existing agent-based resource advertisement and discovery system. This paper presents an 
agent system which implements the GRid Information Protocol (GRIP) and the GRid 
Registration Protocol (GRRP) of the MDS to discover virtual organisations and monitor their 
respective resources. The resulting system has the effect of resource brokering, monitoring and 
performance prediction, 

@ ^ Application Response Measurement of Distributed Web Services 
J.D.Tumer, D.Bacigalupo, S.A.Jarvis, D.N.Dillenberger, G.R.Nudd 
CMG International Journal of Computer Resource Management, 108, pp. 45-55, 2002 

Web service technology will provide a platform for next generation dynamic e-business 
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applications. This paper describes a framework for identifying, monitoring and reporting 
performance data of critical transactions within a web service using the Java ARM standard, a 
Transaction Definition Language (TDL) and a bytecode instrumentation tool. The data extracted 
using this framework can be used as part of a service selection and routing algorithm, an 
example of which is described in this paper. A case study is provided using Gourmet2Go, a 
demonstrator suppUed with the IBM web services toolkit. The results show how transaction- 
based performance monitoring can be used as a basis for dynamic service routing. 

^ ARMS: an Agent-based Resource Management System for Grid Computing. 

J.Cao, S.A.Jarvis, S.Saini, D.J.Kerbyson and G.R.Nudd. 

To appear in Scientific Programming, Special Issue on Grid Computing. 

lOS Press, 2002. 

Resource management is an important component of a grid computing infrastructure. The 
scalability and adaptability of such systems are two key challenges that must be addressed. In 
this work an agent-based resource management system, ARMS, is implemented for grid 
computing. ARMS utilises the performance prediction techniques of the PACE toolkit to 
provide quantitative data regarding the performance of complex applications running on a local 
grid resource. At the meta-level, a hierarchy of homogeneous agents are used to provide a 
scalable and adaptable abstraction of the system architecture. Each agent is able to cooperate 
with other agents and thereby provide service advertisement and discovery for the scheduling of 
applications that need to utilise grid resources. A case study with corresponding experimental 
results is included to demonstrate the efficiency of the resource management and scheduling 
system. 

Q © Agent-based Resource Management for Grid Computing 

J. Cao, D. P. Spooner, J. D. Tumer, S. A. Jarvis, D. J. Kerbyson, S. Saini, and G. R. Nudd. 
2nd IEEE International Symposium on Cluster Computing and the Grid, 
Beriin, Germany, May 2002. 

It is envisaged that the grid infrastructure will be a large-scale distributed software system that 
will provide high-end computational and storage capabilities to differentiated users. A number 
of distributed computing technologies are being applied to grid development work, including 
CORBA and Jini. In this work, we introduce an A4 (Agile Architecture and Autonomous 
Agents) methodology, which can be used for resource management for grid computing. An 
initial system implementation utilises the performance prediction techniques of the PACE 
toolkit to provide quantitative data regarding the performance of complex applications nmning 
on local grid resources. At the meta-level, a hierarchy of identical agents is used to provide an 
abstraction of the system architecture. Each agent is able to cooperate with other agents to 
provide service advertisement and discovery to schedule applications that need to utilise grid 
resources. A performance monitor and advisor (PMA) is in development to optimise the 
performance of agent behaviours. 

Q Performance Prediction Technology for Agent-based Resource Management 

J. Cao, S. A. Jarvis, D. P. Spooner, J. D. Tumer and G. R. Nudd. 
Uth IEEE Heterogeneous Computing Workshop (HCW02), 
Marriott Marina, Fort Lauderdale, Florida. 15- 19th April 2002. 

Resource management constitutes an important infrastructural component of a computational 
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grid environment. The aim of grid resource management is to efficiently schedule applications 
over the available resources provided by the supporting grid architecture. Such goals within the 
high performance community rely, in part, on accurate performance prediction capabilities. 

This paper introduces a resource management infrastructure for grid computing environments. 
The technique couples application performance prediction with a hierarchical multi-agent 
system. An initial system implementation utilises the performance prediction capabilities of the 
PACE toolkit to provide quantitative data regarding the performance of complex applications 
running on local grid resources. The validation results show that a high level of accuracy can be 
obtained, that cross-platform comparisons can be easily undertaken, and that the estimates can 
be evaluated rapidly. A hierarchy of homogeneous agents are used to provide a scalable and 
adaptable abstraction of the grid system architecture. An agent is a representative of a local grid 
resource and is considered to be both a service provider and a service requestor. Agents are 
organised into a hierarchy and cooperate to provide service advertisement and discovery. A 
performance monitor and advisor has been developed to optimise the performance of the agent 
system. A case study with corresponding experimental results are included to demonstrate the 
efficiency of the resource management and scheduling system. 

The main features of the system include: hard quality of service support using PACE 
performance prediction capabilities; agent-based dynamic resource advertisement and discovery 
capabilities; simulation-based quantitative grid performance analysis and user-oriented 
scheduling of local grid resources. 

^ ^ A Transaction Definition Language for Java Application Response Measurement 
J.D. Turner, D.P. Spooner, J. Cao, S.A. Jarvis, D.N. Dillenberger, G.R. Nudd. 
27th International Conference on Technology Management and Performance Evaluation of 
Enterprise-Wide Information Systems (CMG2001) 
2-7 December 2001, Anaheim, Califomia, 

Published in Journal of Computer Resource Measurement, 105, pp.55-65, 2001. 

This paper describes a technique for automatically ARMing Java applications in accordance 
with the ARM 3.0 standard for Java (submitted to the Open Group February 2001). An XML- 
based Transaction Definition Language is defined, which allows a developer to semantically 
define performance critical components of Java applications. An application is instrumented 
with ARM method calls through a bytecode transformer prior to execution, providing ARM 
compliance while removing the necessity to modify (or even possess) the original Java source 
code 

S ® High Performance Service Discovery in Large-Scale Multi-Agent and Mobile- Agent 
Systems. 

J. Cao, D.J. Kerbyson, and G.R. Nudd. 

International Journal of Software Engineering and Knowledge Engineering, Special Issue on 

Multi-Agent Systems and Mobile Agents, 

World Scientific Publishing, 1 1(5), 621-641, 2001. 

With increasing requirements of distributed software systems, software agents are becoming a 
mainstream technology for software engineering and data management. Scalability and 
adaptability are two key challenges that must be addressed. In this work a new model is 
introduced for building large-scale distributed software systems with high dynamics, using a 
hierarchy of homogeneous agents that has the capability of service discovery. The performance 
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of the agent system can be improved using di erent combinations of optimisation strategies. A 
modelling and simulation environment has been developed to aid the performance evaluation 
process. Two case studies are given and simulation results are included that show the impact of 
the agent mobility and the choice of performance optimisation strategies on the overall system 
performance. 

^ Use of Agent-based Service Discovery for Resource Management in Metacomputing 
Environment. 

J. Cao, D. J. Kerbyson, and G. R. Nudd. 

In Proceedings of 7th International Euro-Par Conference. 

Manchester, UK, Lecture Notes in Computer Science 2150, Springer Verlag, 882-886, August 
2001. 

A new methodology is presented in this paper for resource management in a metacomputing 
environment using a hierarchy of homogeneous agents that has the capability of service 
discovery. The PACE tools are used to provide quantitative data concerning the performance of 
sophisticated applications running on local high performance resources. At metacomputing 
level, an agent hierarchy is used to model the resource management system, and the 
implementation of resource management, scheduling, and allocation can be abstracted to the 
processes of service advertisement and service discovery. Different optimisation strategies can 
be used to improve the performance of the agent system. 

Q "Wi Application Characterisation using a Lightweight Transaction Model 

D.P. Spooner, J.D. Tumer, J. Cao, S.A. Jarvis, G.R. Nudd. 

In proc. of UK Performance Engineering Workshop Leeds 2001, pp. 215-225, 

With the emergence of GRID environments and distributed systems with dynamic resources and 
varying user profiles, there is an increasing need to develop reliable tools that can ef-fectively 
coordinate the requirements of an application with available computing resources. The ability to 
predict the behaviour of complex aggregated systems under dynamically changing workloads is 
particularly desirable, leading to effective resource usage and opti-misation of networked 
systems. 

To explore these issues, the High Performance Systems Group (HPSG) at Warwick are 
developing a resource management architecture called PACE Lite that builds upon an exist-ing 
set of research tools used in the area of performance prediction. The system embodies the 
concept of a transaction that encapsulates a key application component. Using software sensors, 
user requirements, system policies and resource usage histories, an application is rapidly 
characterised and scheduled appropriately. A distributed agent-based environment is used as the 
system infrastructure, and scheduling optimisation is achieved by a genetic algorithm that 
attempts to minimise the make-span across systems. 

€ W Dynamic Optimisation of Application Execution on Distributed Systems. 
A.M. Alkindi, D.J. Kerbyson, E. Papaefstathiou, and G.R. Nudd. 
Future Generation Computing Systems, Vol. 17, No. 8, Elsvier, June 2001, pp. 941-949. 

In this paper, we demonstrate the impact of a using dynamic (on-the-fly) performance 
prediction tool-set, PACE, for optimising application execution on dynamic systems. The need 
for steering the application execution arises from the ever-growing use of distributed and GRID 
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systems. The unquestionable aim to overcome bottleneck problems, allocation, and performance 
degradation due to shared CPU time has prompted many investigations into the best way in 
which the performance of an application can be enhanced. In this work, we present a novel 
approach to dynamically optimise the performance of an application. An example application, 
the FFTW (the fastest Fourier transform in the west), is used to illustrate the approach which 
itself is a novel method that optimises the execution of an FFT. It is shovra that the performance 
prediction can provide the same quality of information as a measurement preocess for 
application optimisation but in a fraction of the time and thus improve overall application 
performance. 

@ Dynamic Instrumentation and Performance Prediction of Application Execution. 
A.M. Alkindi, D.J. Kerbyson, and G.R. Nudd. 

In Proc. of High Performance Computing and Networking (HPCN2001), 

Lecture Notes in Computer Science, Vol. 2110, Springer- Verlag, Amsterdam, June 2001, pp. 

313-323. 

This paper presents a new technique that enhances the process and the methodology used in a 
performance prediction analysis. An automatic dynamic instrumentation methodology is added 
to Warwick's Performance Analysis and Characterization Environment PACE. The automation 
process has eluninated the need to manually obtain application information and data. The 
dynamic instrumentation has given PACE the ability to extract and utilize data that were hidden 
and unobtainable prior to execution. We give two examples to illustrate our methodology. 
While it was impossible to perform the analysis using the original method due to lack of 
essential information, the new technique successfully enabled PACE to conduct the prediction 
analysis in a dynamic environment. The results show that with the automated dynamic 
instrumentation, the performance predication analysis of dynamic application execution is 
possible and the results obtained are reliable. We believe that the technique implemented here 
could eventually be used in other performance prediction tool-sets, and therefore enhance the 
ways in which the performance of systems and applications is analysed and predicted. 

^ Performance Evaluation of an Agent-Based Resource Management Infrastructure for 
Grid Computing 

J. Cao, D.J. Kerbyson, G.R. Nudd. 

In Proceedings of 1st IEEE/ ACM International Symposium on Cluster Computing and the Grid. 
Brisbane, Australia, 311-318, May 2001. 

Resource management is an important infrastructure in the grid computing environment. 
Scalability and adaptability are two key challenges in the implementation of such complex 
software systems. In this work we introduce a new model for resource management in a 
metacomputing environment using a hierarchy of homogeneous agents that has the capabiUty of 
service discovery. The performance of the agent system can be improved using different 
combinations of optimisation strategies. A modelling and simulation environment has been 
developed in this work that enables the performance of the system to be investigated. A 
simplified model of the resource management infrastructure is given as a case study and 
simulation results are included that show the impact of the choice of performance optimisation 
strategies on the overall system performance. 

^ ^ Realistic Image Synthesis of Plant Structures for Genetic Analysis 
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D, Wang, DJ. Kerbyson, G J. King and G.R. Nudd. 

Image and Vision Computing, Vol. 19, No. 8, Elsevier, May 2001, pp. 517-522. 

The objective of this study was to develop a systematic method to restore images of plants using 
a small number of parameters in order to facilitate investigation of the genetic behaviour of 
branching patterns by computer models. Based on biological data, a generic methodology for 
image synthesis of plant structures was proposed. The corresponding algorithms for feature 
extraction and tuning were designed. Images of plant structures were synthesised with a higher 
accuracy by a model that was constructed using parametric L-systems. The algorithms were 
validated by comparing the synthetic images with the real plants. The approach makes it 
possible to assess the virtual architectures of plants by computer models, instead of real 
experiments, and to investigate the relationship between plant morphology and their genetic 
behavious. This may aid the understanding of complex branching patterns that result from a 
plant's genetic behaviour. 

@ ^ Is Predictive Tracing Too Late for HPC Users? 

D J. Kerbyson, E. Papaefstathiou, J.S. Harper, S.C. Perry, G.R. Nudd. 

In: RJ, Allan, ASimpson, and DA, Nicole, eds,, High Performance Computing, Plenum Press, 
1998. 

An underlying goal in the use of high performance systems is to apply the often quite complex 
systems to achieve rapid application execution times. The analysis of performance is currently 
an active research area. Traditionally performance prediction has been used prior to application 
implementation to explore alternative implementations e.g. different parallelisations, and also in 
analysing existing codes in order to access the impact of increasing resources, and also to 
identify bottlenecks. 

The aim of a novel prediction system currently under development at Warwick, is to extend the 
traditional use of performance prediction to encapsulate the full software cycle. It incorporates 
both pre and post-implementation analysis thus allowing alternatives to be explored prior to 
committing an application (and its parallelisation) to a system, or to refine an existing 
implementation. The approach is carefiiUy structured with modular performance models to 
reflect parts of the whole system (e.g. software components, parallelisation components, and 
hard ware/sy stem performances). 

In the work to be presented here, it will be shown how this system is capable of producing 
predictive traces which allow a detailed analysis of the system performance prior to actual 
execution. The trace data output from the system is in the PICL format allowing an HPC user to 
utilise familiar performance monitoring tools. Thus, application execution can be easily viewed, 
and performance hotspots identified from detailed performance predictions. 

An example of Financial Option codes will be utilised to illustrates the benefits of this approach 
on clusters of workstations. The approach has a potentially large applicability in the HPC 
domain when applied across a range of target platforms which also allows inter-platform 
performance comparisons to be made. 

® ^ Application Execution Steering Using On-the-fly Performance Prediction 

D.J. Kerbyson, E. Papaefstathiou, G.R. Nudd. 

In: High Performance Computing and Networking 98, Amsterdam, Holland,April 1998, Lecture 
Notes in Computer Science, Springer-Verlag. 
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The execution of an application on a high performance system requires parameters concerning 
the problem in hand, and those that determine the system mapping, to be specified by a user. 
The system parameters are typically used to minimise the execution time. However, by the 
coupling of a performance model with an application, system parameters can be determined 
without user intervention. In the work presented here, a novel performance prediction system 
has been used to provide suitable performance models which can determine appHcation 
mapping parameters, code execution decisions, and system choices on-the-fly. An example 
compact application of a convolution is used to illustrate the approach for automatically 
choosing the actual code to be executed, and the number of workstations in a cluster to be 
utilised. The performance prediction system is shovm to have a reasonable accuracy 
(approximately 10% error), with a rapid evaluation time (typically less than 2s). 



^ ^ Predicting the Cache Miss Ratio of Loop-Nested Array References 

J.S. Harper, DJ. Kerbyson, G.R. Nudd. 

Research Report 336, Department of Computer Science, University of Warwick, Coventry, UK, 
December 1997. 

The time a program takes to execute can be massively affected by the efficiency with which it 
utilizes cache memory. Moreover the cache-miss behavior of a program can be highly 
unpredictable, in that small changes to input parameters can cause large changes in the number 
of misses. In this paper we present novel analytical models of the cache behavior of programs 
consisting mainly of array operations inside nested loops, for direct-mapped caches. The models 
are used to predict the miss-ratios of three example loop nests; the results are shovra to be 
largely within ten percent of simulated values. A significant advantage is that the calculation 
time is proportional to the number of array references in the program, typically several orders of 
magnitude faster than traditional cache simulation methods. 

@ ^ An Introduction to the Layered Characterisation for High Performance Systems 
E. Papaefstathiou, D.J. Kerbyson, G.R. Nudd, TJ. Atherton, J.S. Harper. 
Research Report 335, Department of Computer Science, University of Warwick, Coventry, UK, 
December, 1997, 

A toolset for performance analysis of parallel systems, PACE, is presented in this report. In this 
toolset expert knowledge about the performance evaluation techniques is not required as a 
prerequisite for the user. Instead a declarative approach to the performance study is taken by 
describing the application in a way that is both intuitive to the user, but can also be used to 
obtain performance results. The underlying performance related characterisation models and 
their evaluation processes are hidden firom the user. This docxmient describes the special 
purpose language, and the evaluation system, that form the core of the PACE toolset. Amongst 
the aims of the toolset is the support of characterisation model reusability, ease of 
experimentation, provide different levels of prediction accuracy, and support of different levels 
of characterisation model abstraction. 

^ ^ PACE: A Toolset to Investigate and Predict Performance in Parallel Systems 
D.J. Kerbyson, J.S. Harper, A. Craig, G.R Nudd. 

Presented in European Parallel Tools Meeting, ONERA, Paris, October 1996, 

A toolset, PACE for predicting and analysing performance issues in parallel systems is 
presented here. In this toolset expert knowledge about performance evaluation techniques is not 
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required as a prerequisite for its use. Instead, an intuitive approach is taken by describing the 
application under study in way that is accessible to the user. A modular approach is taken which 
permits separate descriptions of sub-tasks, parallelisation strategies, and platform descriptions to 
take place. PACE has been used to predict performance for number of applications on a number 
of different parallel platforms. Results to date have indicated that the predictions typically 
within 5-7% of actual measurements. 
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A system model is a tool used to predict system performance under changing conditions. 
There are two widely used modeling techniques: one based on discrete event simulation and 
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in them. Users are developing and using queuing theory models to project system 
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Papaefstathiou, G.R. Nudd, in Proc. UK Performance Engineering Workshop (UKPEW'99), Bristol, 
July 1999, pp. 413-424. 
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Workshop, Daresbury Labs, UK, May 1998. 
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